refactor unicsv & xcsv date time handling (#1114)
authortsteven4 <13596209+tsteven4@users.noreply.github.com>
Tue, 16 May 2023 21:44:47 +0000 (15:44 -0600)
committerGitHub <noreply@github.com>
Tue, 16 May 2023 21:44:47 +0000 (15:44 -0600)
commit4aa594a348231eb90ccb20bd7a140a949786bc7a
treedba02a7f23713d07d234003bc4056e4ac5166607
parent488aa41264c3c94c8c6e1d3de41ebd0c54d0c0d9
refactor unicsv & xcsv date time handling (#1114)

* refactor unicsv date time handling.

* restrict utc option range to match Qt offsetfromutc

* move xcsv from C-style legacy time to Qt.

* fill in lower order date/time fields when parsing.

* don't return date/time if we don't have one!

* enhance xcsv date/time testing, fix bug.

* csv format date time adjustements.

use QDateTime::fromString to parse iso date times.
return invalid QTime from addhms if parsing fails entirely.

* datetime display fixes.

For the xcsv writer:
avoid priting date/time fields when there isn't a valid date/time.
when printing with am/pm use times from a 12 hour clock.
add support for printing dotnet time.
use QDateTime::toString to print ISO time.
For the xcsv and unicsv readers:
use QDateTime::fromString instead of xml_parse_time when reading
ISO datetimes.  This avoids xml_parse_times intentional odd behavior
of treating non-timezoned times as UTC.

add a test of the xcsv writer time related fields.  This only runs
in America/Denver time zone.  As setting the time zone is system
dependent the test only runs if tzselect is available.  If so it
assumes America/Denver is available.

correct documentation to give a sensible format for xcsv field GMT_TIME.

* add reference files for new test.

* silence xcsv reader conversion warnings on empty strings.

* warn on parse errors reading excel time.

* xcsv date/time fixes.

fix addhms to account for 12/24 hour clock.
don't print invalid datetimes with iso_time, iso_time_ms.

add testcases to exercise all the xcsv reader date/time flavors.

* clarify HMSL, HMSG wrt 12/24 hour clocks.

* add missing reference file.

* enhance xcsv for HMS[L|G] before or after [LOCAL|GMT]_TIME

* restore YYYYYMMDD to use UTC.

This has been broken for some time.  Mail from 2012 indicates the intent
was UTC (https://sourceforge.net/p/gpsbabel/mailman/message/29544538/)

* unicsv review catches

don't ripple into trouble with msec rounding.
pass outputs that may or may not be written as references.

* retire xcsv fields HMSG_TIME, HMSL_TIME.

This is potentially a user visible change.  It could require
users to rewrite any style files they have created that use
these fields.

These are replaced by repeated use of GMT_TIME and LOCAL_TIME.
This eliminates our pain over 12/24 hour clock issues. strptime/
strftime (as well as QTime) have distinct conversion specifiers
for hours with 12/24 hour clocks.  Our support for HMSG_TIME,
HMSL_TIME used integer conversion specifiers for hours, minutes
and seconds.  This made it difficult to decide if a 12 or 24 hour
clock should be used, and made it impossible to have reduced
precision values with 12 hour clock using an AM/PM designation.
We also always printed AM/PM designations.

* fix whitespace in serialization reference file
31 files changed:
defs.h
reference/datetime.xcsv [new file with mode: 0644]
reference/datetime_read.xcsv [new file with mode: 0644]
reference/datetime~xcsv.xcsv [new file with mode: 0644]
reference/format3.txt
reference/help.txt
reference/localgmttime.csv [new file with mode: 0644]
reference/localgmttime.xcsv [new file with mode: 0644]
reference/realtime.csv
reference/unicsv-local-utc.gpx [new file with mode: 0644]
reference/unicsv-local-utc1.gpx [new file with mode: 0644]
reference/unicsv-local.csv [new file with mode: 0644]
reference/unicsv-local.gpx [new file with mode: 0644]
reference/unicsv-local~csv.csv [new file with mode: 0644]
reference/unicsv-local~gpx.csv [new file with mode: 0644]
reference/unicsv-utc.csv [new file with mode: 0644]
reference/unicsv-utc.gpx [new file with mode: 0644]
style/garmin_g1000.style
style/iblue747.style
style/iblue757.style
style/land_air_sea.style
testo.d/track.test
testo.d/unicsv_local.test [new file with mode: 0644]
testo.d/xcsv.test
unicsv.cc
unicsv.h
util.cc
xcsv.cc
xcsv.h
xmldoc/chapters/styles.xml
xmldoc/formats/options/unicsv-utc.xml